Geographic Topic Model: Appendix

Faceted topic models combine topical content with extraneous facets, such as ideology or dialect. In this model, the “pure” topics are corrupted by the facets, using a hierarchical generative model in which the pure topics act as priors on the faceted topics. This is most easily modeled using the logistic-normal distribution, which admits a normal prior on the mean. 1 Model We build on latent Dirichlet allocation: • For each document d, draw θd ∼ Dirichlet(αθ) • For each token n < Nd – Draw a topic index zn ∼ θd – Draw a word token from the associated topic wn ∼ βzn We augment each document with an additional discrete facet variable, vd ∼ θ, which selects the appropriate faceted topic (θ ∼ Dir(αθ)): wn ∼ βd zn . Here zn indexes the topic and vd indicates the facet of the topic: for example, vd may select the “Detroit” version of the “electronic music” topic. We may also introduce metadata yd, such that yd ∼ f(y;ρvd), indicating an arbitrary probability distribution with parameters ρvd . In the geographical topic model, f(y;ρvd) takes the form of a bivariate Gaussian; variational inference in this setting is deferred to [4]. The faceted topics take logistic-Normal priors, such that β k ∼ LN (μk,σk). It will be convenient to introduce the auxiliary variable η, such that β = expη/ ∑ i exp η[i] and η (j) k ∼ N (μk,σk). Throughout, i will index the word, j will index the facet, and k will index the topic. We draw the pure topics from Normal priors, μk ∼ N (a, b)), and the topic-variances from Gamma priors, σ k[i] ∼ G(c, d). 2 Variational Approximation We’ll make a fully-factorized approximation: Q(z,v,θ,θ,η,μ,σ) =  K ∏